feat: add TUI agent harness with MCP server by aidandaly24 · Pull Request #548 · aws/agentcore-cli

aidandaly24 · 2026-03-17T13:38:58Z

Description

Adds a TUI agent harness that enables AI agents (Claude Code, Kiro) to programmatically interact with the AgentCore CLI's Terminal UI through headless pseudo-terminals.

Architecture:

Core library (src/tui-harness/lib/) — TuiSession class wrapping node-pty + @xterm/headless for headless terminal emulation. Provides sendKeys, sendSpecialKey, readScreen, waitFor, and close operations. Includes settling detection (waits for terminal output to stabilize), session management with signal-handler cleanup, and screen reading utilities.
MCP server (src/tui-harness/mcp/) — Exposes 7 tools (tui_launch, tui_send_keys, tui_read_screen, tui_wait_for, tui_screenshot, tui_close, tui_list_sessions) over stdio transport using @modelcontextprotocol/sdk. Built as a standalone binary at dist/mcp-harness/index.mjs.
Documentation — AGENTS.md updated with complete harness usage guide including screen identification markers, a 27-step create wizard example captured from ground truth, navigation patterns, and error recovery guidance.

Key design decisions:

@xterm/headless marked as external in esbuild (CJS-only package, bundling mangles default export)
SpecialKey type derived from SPECIAL_KEY_VALUES const array (single source of truth)
WaitForTimeoutError returns {found: false} from MCP layer, not isError — lets agents decide next action
tui_launch({}) defaults to node dist/cli/index.mjs for zero-config AgentCore CLI launching

Related Issue

Closes #

Documentation PR

N/A — documentation is inline in AGENTS.md and docs/TESTING.md

Type of Change

Testing

How have you tested the change?

I ran npm run test:unit and npm run test:integ
I ran npm run typecheck
I ran npm run lint
If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Additional verification:

Both esbuild bundles (CLI + MCP harness) build successfully
MCP server starts and responds to stdio transport
Proof-of-concept unit tests pass (xterm standalone, PTY+xterm wiring, DSR/CPR handler)
Integration tests (27 tests across 5 suites) pass in follow-up PR

Checklist

I have read the CONTRIBUTING document
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

github-actions · 2026-03-17T13:43:06Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	41.69%	4175 / 10013
🔵	Statements	41.34%	4414 / 10677
🔵	Functions	42.44%	789 / 1859
🔵	Branches	44.04%	2783 / 6318

Generated in workflow #1021 for commit 0b50a6b by the Vitest Coverage Report Action

jesseturner21

Just one nit comment about the agents.md file

Headless terminal harness using node-pty + @xterm/headless that spawns real CLI processes in a PTY and reads screen state programmatically. Core components: - TuiSession: spawn, sendKeys, sendSpecialKey, readScreen, waitFor, close - SettlingMonitor: text-content comparison to filter cursor blink - Screen reader: viewport/scrollback reading, numbered output - Key map: named keys to escape sequence mapping - Session manager: global registry with process-exit cleanup - Availability check: graceful skip when node-pty is missing waitFor() throws WaitForTimeoutError on timeout (not silent return). launch() races settle vs process exit, throws LaunchError on crash. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

7 MCP tools exposed via stdio transport for AI agents to drive the TUI: tui_launch, tui_send_keys, tui_read_screen, tui_wait_for, tui_screenshot, tui_close, tui_list_sessions. Session map with max 10 concurrent sessions. tui_wait_for catches WaitForTimeoutError and returns {found: false} (not an MCP error). tui_launch defaults to AgentCore CLI when no command specified. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- devDependencies: @xterm/headless, @modelcontextprotocol/sdk - optionalDependencies: node-pty (native addon, graceful skip) - esbuild: second entry point for mcp-harness bundle - vitest: new 'tui' project with fileParallelism: false - .mcp.json: MCP server discovery for Claude Code - package.json: bin entry + test:tui script Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

AGENTS.md: TUI harness section with MCP tool reference, complete 27-step create wizard example (verified against real TUI), screen identification markers table, screenshot format, error recovery patterns, navigation patterns, and known limitations. TESTING.md: TUI integration test guide with TuiSession API reference, ScreenState type, special keys list, waitFor vs settling guidance, WaitForTimeoutError output example, and LaunchError handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Move harness code from two separate directories (src/test-utils/tui-harness/ and src/mcp-harness/) into a single src/tui-harness/ directory with lib/ and mcp/ subdirectories. Also cleans up dead code in tools.ts, derives SpecialKey type from SPECIAL_KEY_VALUES array (single source of truth), and fixes cross-boundary import of createMinimalProjectDir. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The harness is dev-only tooling for AI agents and integration tests. It should not ship to end users who install the CLI. - Gate MCP harness esbuild behind BUILD_HARNESS=1 env var - Remove agent-tui-harness bin entry from package.json - Add !dist/mcp-harness to files array (npm publish exclusion) - Remove node-pty from optionalDependencies (stays in devDependencies) - Add build:harness script, update test:tui to use it - Update AGENTS.md to reference npm run build:harness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Reduces AGENTS.md context overhead for agents that don't need TUI harness details. Leaves a one-line pointer to the full guide in docs/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

tejaskash

LGTM

Aligns with the kebab-case convention used by other files in src/tui-harness/lib/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

aidandaly24 requested a review from a team March 17, 2026 13:38

aidandaly24 temporarily deployed to e2e-testing March 17, 2026 13:39 — with GitHub Actions Inactive

github-actions Bot added the size/xl PR size: XL label Mar 17, 2026

aidandaly24 mentioned this pull request Mar 17, 2026

feat: add TUI integration tests #549

Closed

15 tasks

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 17, 2026

aidandaly24 temporarily deployed to e2e-testing March 17, 2026 19:15 — with GitHub Actions Inactive

jesseturner21 previously approved these changes Mar 17, 2026

View reviewed changes

Comment thread AGENTS.md Outdated

jesseturner21 dismissed their stale review via 5e2f90b March 17, 2026 23:01

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 17, 2026

jesseturner21 temporarily deployed to e2e-testing March 17, 2026 23:01 — with GitHub Actions Inactive

aidandaly24 and others added 6 commits March 18, 2026 12:05

aidandaly24 force-pushed the feat/tui-harness branch from 5e2f90b to 1541fc3 Compare March 18, 2026 16:05

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 18, 2026

aidandaly24 temporarily deployed to e2e-testing March 18, 2026 16:05 — with GitHub Actions Inactive

docs: move TUI harness guide from AGENTS.md to docs/tui-harness.md

ec15d99

Reduces AGENTS.md context overhead for agents that don't need TUI harness details. Leaves a one-line pointer to the full guide in docs/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 18, 2026

aidandaly24 temporarily deployed to e2e-testing March 18, 2026 16:16 — with GitHub Actions Inactive

style: fix prettier formatting in AGENTS.md

60db522

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

aidandaly24 temporarily deployed to e2e-testing March 18, 2026 17:44 — with GitHub Actions Inactive

github-actions Bot removed the size/xl PR size: XL label Mar 18, 2026

github-actions Bot added the size/xl PR size: XL label Mar 18, 2026

tejaskash previously approved these changes Mar 18, 2026

View reviewed changes

Merge branch 'main' into feat/tui-harness

3e0471b

jesseturner21 dismissed tejaskash’s stale review via 3e0471b March 18, 2026 18:12

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 18, 2026

jesseturner21 temporarily deployed to e2e-testing March 18, 2026 18:13 — with GitHub Actions Inactive

Hweinstock reviewed Mar 18, 2026

View reviewed changes

Comment thread src/tui-harness/lib/tui-session.ts

Comment thread docs/tui-harness.md

jesseturner21 previously approved these changes Mar 19, 2026

View reviewed changes

style: rename TuiSession.ts to kebab-case tui-session.ts

0b50a6b

Aligns with the kebab-case convention used by other files in src/tui-harness/lib/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jesseturner21 dismissed their stale review via 0b50a6b March 19, 2026 12:23

jesseturner21 temporarily deployed to e2e-testing March 19, 2026 12:23 — with GitHub Actions Inactive

github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Mar 19, 2026

Hweinstock approved these changes Mar 19, 2026

View reviewed changes

jesseturner21 merged commit c51b1e2 into main Mar 19, 2026
18 of 19 checks passed

jesseturner21 deleted the feat/tui-harness branch March 19, 2026 12:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add TUI agent harness with MCP server#548

feat: add TUI agent harness with MCP server#548
jesseturner21 merged 10 commits into
mainfrom
feat/tui-harness

aidandaly24 commented Mar 17, 2026

Uh oh!

github-actions Bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

jesseturner21 left a comment

Uh oh!

Uh oh!

tejaskash left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

aidandaly24 commented Mar 17, 2026

Description

Related Issue

Documentation PR

Type of Change

Testing

Checklist

Uh oh!

github-actions Bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report

Uh oh!

jesseturner21 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tejaskash left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Mar 17, 2026 •

edited

Loading